
*	************************************************************************
* 	File-Name: 	main.do
*	Log-file:	n/a
*	Date:  		7 November 2019
*	Author:		Patrick Bayer
*	Paper:		Patrick Bayer, Ryan Kennedy, Joonseok Yang, and Johannes Urpelainen
*				The Need For Impact Evaluation in Electricity Access Research
*	Journal:	Energy Policy
*	Data Used:  ./Impact_data.xls
*	Output		n/a
*	Purpose:   	file prepares raw data for analysis
*	************************************************************************


*	************************************************************************
*   Techincal disclaimer:
*   All files/ data were created/ analysed in Stata/SE 14.2 on Windows 10, 64 bit
*	Machine: Intel(R) Core(TM) i7-4650U CPU @ 1.70Ghz 2.30Ghz with 8GB RAM
*	************************************************************************


clear all
set more off, permanently

*	************************************************************************
*	A. Load data and setup
*	************************************************************************

do "./prep.do"


*	************************************************************************
*	B. Main description of data
*	************************************************************************

preserve
collapse (first) Method MethodDetail Technology Source Referred Country, by(StudyID)

tab Source
tab Referred
restore

tab Impact

* Table 1: Direct and indirect measures of outcomes
tab Outcome if Impact!=.
sort Variable
by Variable: tab Outcome if Impact!=.

* Table 2: Coding of study method
preserve
collapse (first) MethodDetail, by(StudyID)
tab MethodDetail
restore



*	************************************************************************
*	C.1 Results section 
*	************************************************************************

* Note: Unit-of-analysis is study-outcome, with each study having 5 outcomes each
tab StudyID
di 31/19
tab StudyID if Year>2013
tab Outcome if Impact!=.
tab SpecificMeasure if Outcome==2 & Impact!=.


*	************************************************************************
*	C.2 Results section (observational vs experimental)
*	************************************************************************

* Figure 1: Mosaic plot (full data)
tab Impact
tab Method
tabulate Impact Method, chi2
di 37/44
di 11/21


* Figure 2: Mosaic plot (by grid/offgrid)
preserve
collapse (first) Technology Method, by(StudyID)
tab Technology
ttest Technology, by(Method)
tabulate Technology Method, chi2
reg Technology Method, cluster(StudyID)
boottest Method
boottest Method, cluster(StudyID) weighttype(normal) level(95)
restore

sort Technology
by Technology: tabulate Impact Method, chi2
di 29/34
di 2/5
di 8/10
di 9/16
 
* See appendix section below for code for regression models

* Figure 3: Bar plot impact by outcome

tab Outcome if Impact!=.
di (22+19)/65
sort Outcome
by Outcome: tab Method Impact


*	************************************************************************
*	C.3 Results section (all four method types)
*	************************************************************************

tab MethodDetail Impact, chi2
di 15/17
di 10/11
di 12/16
di 11/21
di 48/65

table MethodDetail
di 17/10
di 11/8
di 16/6
di 21/7

* See appendix section below for code for regression models

*	************************************************************************
*	C.3 Results section (alternative explanations) 
*	************************************************************************

* Ambition level analysis
tab Outcome if Impact!=.
gen Ambition=0
replace Ambition=1 if Outcome==2 | Outcome==3

sort Ambition
by Ambition: tab Method Impact, chi2
di 15/21
di 3/9
di 22/23
di 8/12
by Ambition: tab MethodDetail Impact, chi2

* African vs non-African region
tab Country
gen Africa=0
replace Africa=1 if Country=="Kenya" | Country=="Nigeria" | Country=="Rwanda" | Country=="Senegal" ///
| Country=="South Africa" | Country=="Tanzania" | Country=="Uganda"

sort Africa
by Africa: tab Method Impact, chi2
di 26/30
di 4/8
di 11/14
di 7/13
by Africa: tab MethodDetail Impact, chi2
by Africa: tab Impact



*	************************************************************************
*	D. Appendix
*	************************************************************************

*	************************************************************************
*	D.1 List of studies (SI2)
*	************************************************************************

/* Write output table for list of studies
keep StudyID Study Year Technology Country Method MethodDetail
collapse (first) Study Year Technology Country Method MethodDetail, by(StudyID)
label value Method mlabel
label value MethodDetail mdlabel
label value Technology tlabel
browse
export delimited using "./StudyList.txt", delimiter("&") replace
*/


*	************************************************************************
*	D.2 List of impact coding (SI3)
*	************************************************************************

/* Write output table for impact coding
keep ID StudyID Study Year MethodDetail Technology Outcome Impact PointEstimates StatisticalSignificance SpecificMeasure
drop if Impact==.
export delimited using "./ImpactCodingList.txt", delimiter("&") replace
*/


*	************************************************************************
*	D.3 Sample description (SI4)
*	************************************************************************

* Section SI4.1: Studies by country
* Table SI5:
preserve
collapse (first) Country MethodDetail Technology, by(StudyID)
label value MethodDetail mdlabel
label value Technology tlabel
sort Country
by Country: tab MethodDetail Technology
tab MethodDetail
tab Technology
restore

* Section SI4.2: Studies over time
* Figure SI1:
preserve
collapse (first) Year Method, by(StudyID)
tab Year
restore

* Section SI4.3: Studies by technology
preserve
collapse (first) Year Technology Method MethodDetail Referred, by(StudyID)
label value Method mlabel
label value MethodDetail mdlabel
label value Technology tlabel

tab Method
tab MethodDetail if Method==1
tab Year
tab Method if Year==2017

* Table SI6:
tab MethodDetail Technology
tab MethodDetail Referred

ttest Method, by(Technology)
tab Method Technology, chi2
tab MethodDetail Technology, chi2

ttest Method if Referred==1, by(Technology) 
tab Method Technology if Referred==1, chi2
tab MethodDetail Technology if Referred==1, chi2
restore

* Section SI4.4: Variable coverage
* Figure SI.2:
tab Outcome Method if Impact!=.

tab MethodDetail Outcome if Impact!=., chi2
tab Method Outcome if Impact!=., chi2

*	************************************************************************
*	D.4 Regression results (SI5)
*	************************************************************************

* OLS models for observational vs experimental methods
gen MethodNew = Method-1
label define methodnewl 0 "Observational" 1 "Experimental"
label value MethodNew methodnewl

label variable MethodNew "Experimental method"
gen offgrid=Technology-1
label variable offgrid "Offgrid technology"


* Different model specifications
eststo clear
eststo: reg Impact MethodNew offgrid if Impact >=0, cluster(StudyID)
boottest MethodNew, weighttype(normal) level(90)
eststo: reg Impact MethodNew if  Impact >=0 & offgrid==0, cluster(StudyID)
boottest MethodNew, weighttype(normal) level(90)
eststo: reg Impact MethodNew if Impact >=0 & offgrid==1, cluster(StudyID)
boottest MethodNew, weighttype(normal) level(90)
eststo: reg Impact MethodNew offgrid if Impact >=0 & MethodDetail!=2, cluster(StudyID)
boottest MethodNew, weighttype(normal) level(90)
eststo: reg Impact MethodNew offgrid if Impact >=0 & MethodDetail!=3, cluster(StudyID)
boottest MethodNew, weighttype(normal) level(90)
eststo: reg Impact MethodNew offgrid if Impact >=0 & Variable==2, cluster(StudyID)
boottest MethodNew, weighttype(normal) level(90)
eststo: reg Impact MethodNew offgrid if Impact >=0 & Referred==1, cluster(StudyID)
boottest MethodNew, weighttype(normal) level(90)

/* esttab using "../Manuscript/main.tex", replace b(%9.2f) booktabs stats(N N_clust r2, labels("Observations" "Studies" "$ R^2$") fmt(0 0 3)) /// 
eqlabels(none) drop(_cons ) noconstant se label nonotes legend star(* 0.10 ** 0.05 *** 0.01) ///
mtitles("Full sample" "Grid sample" "Offgrid sample" "w/o IV" "w/o DID" "Strict" "Peer review") ///
addnote ("Dependent Variable: 1=Positive impact; 0=Neutral impact." "Standard errors in parentheses and clustered by study.")
*/


* OLS models for all four method types
gen IV=0
replace IV=1 if MethodDetail==2

gen DIDreg=0
replace DIDreg=1 if MethodDetail==3

gen exp=0
replace exp=1 if MethodDetail==4

label variable IV "IV"
label variable DIDreg "DID"
label variable exp "Experiment"

* Different model specification
eststo clear
eststo: reg Impact IV DIDreg exp offgrid if Impact >=0, cluster(StudyID)
boottest exp, weighttype(normal) level(90)
eststo: reg Impact IV DIDreg exp if Impact >=0 & offgrid==0, cluster(StudyID)
boottest exp, weighttype(normal) level(90)
eststo: reg Impact IV DIDreg exp if Impact >=0 & offgrid==1, cluster(StudyID)
boottest exp, weighttype(normal) level(90)
eststo: reg Impact DIDreg exp offgrid if Impact >=0 & MethodDetail!=2, cluster(StudyID)
boottest exp, weighttype(normal) level(90)
eststo: reg Impact IV exp offgrid if Impact >=0 & MethodDetail!=3, cluster(StudyID)
boottest exp, weighttype(normal) level(90)
eststo: reg Impact IV DIDreg exp offgrid if Impact >=0 & Variable==2, cluster(StudyID)
boottest exp, weighttype(normal) level(90)
eststo: reg Impact IV DIDreg exp offgrid if Impact >=0 & Referred==1, cluster(StudyID)
boottest exp, weighttype(normal) level(90)


/*esttab using "../Manuscript/main_detail.tex", replace b(%9.2f) booktabs stats(N N_clust r2, labels("Observations" "Studies" "$ R^2$") fmt(0 0 3)) /// 
eqlabels(none) drop(_cons ) noconstant se label nonotes legend star(* 0.10 ** 0.05 *** 0.01) ///
mtitles("Full sample" "Grid sample" "Offgrid sample" "w/o IV" "w/o DID" "Strict" "Peer review") ///
addnote ("Dependent Variable: 1=Positive impact; 0=Neutral impact." "Standard errors in parentheses and clustered by study.")
*/



*	************************************************************************
*								END OF DO FILE 
*	************************************************************************



